Critical Points for Least - Squares ProblemsInvolving
نویسنده
چکیده
This paper deals with nonlinear least-squares problems involving the tting to data of parameterized analytic functions. For generic regression data, a general result establishes the countability, and under stronger assumptions niteness, of the set of functions giving rise to critical points of the quadratic loss function. In the special case of what are usually called \single-hidden layer neural networks," which are built upon the standard sigmoidal activation tanh(x) (or equivalently (1 + e ?x) ?1), a rough upper bound for this cardinality is provided as well.
منابع مشابه
Some Convergence Results on the Regularized Alternating Least-Squares Method for Tensor Decomposition
We study the convergence of the Regularized Alternating Least-Squares algorithm for tensor decompositions. As a main result, we have shown that given the existence of critical points of the Alternating Least-Squares method, the limit points of the converging subsequences of the RALS are the critical points of the least squares cost functional. Some numerical examples indicate a faster convergen...
متن کاملLp-Regularized Least Squares (0<p<1) and Critical Path
The least squares problem is formulated in terms of lp quasi-norm regularization (0 < p < 1). Two formulations are considered: (i) an lp-constrained optimization and (ii) an lp-penalized (unconstrained) optimization. Due to the nonconvexity of the lp quasi-norm, the solution paths of the regularized least squares problem are not ensured to be continuous. A critical path, which is a maximal cont...
متن کاملCritical points for least-squares problems involving certain analytic functions, with applications to sigmoidal nets
This paper deals with nonlinear least-squares problems involving the fitting to data of parameterized analytic functions. For generic regression data, a general result establishes the countability, and under stronger assumptions finiteness, of the set of functions giving rise to critical points of the quadratic loss function. In the special case of what are usually called “single-hidden layer n...
متن کاملLeast Squares Support Vector Machine for Constitutive Modeling of Clay
Constitutive modeling of clay is an important research in geotechnical engineering. It is difficult to use precise mathematical expressions to approximate stress-strain relationship of clay. Artificial neural network (ANN) and support vector machine (SVM) have been successfully used in constitutive modeling of clay. However, generalization ability of ANN has some limitations, and application of...
متن کاملIncompressible laminar flow computations by an upwind least-squares meshless method
In this paper, the laminar incompressible flow equations are solved by an upwind least-squares meshless method. Due to the difficulties in generating quality meshes, particularly in complex geometries, a meshless method is increasingly used as a new numerical tool. The meshless methods only use clouds of nodes to influence the domain of every node. Thus, they do not require the nodes to be conn...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 1996